Medical image processing apparatus, medical image processing method, and storage medium

ABSTRACT

A medical image processing apparatus includes an obtaining unit configured to obtain a medical image based on imaging order information, and a determination unit configured to determine, using parameters obtained by machine learning, consistency between the imaging order information and the medical image.

BACKGROUND Field of the Disclosure

The present disclosure relates to a medical image processing apparatus,a medical image processing method, and a storage medium.

Description of the Related Art

In the field of medicine, digitalization of medical images obtained bycapturing images of patients has been implemented. For example,examinations on patients using medical image processing systems(modalities), such as an X-ray examination apparatus, a computedtomography (CT) apparatus, and a magnetic resonance imaging (MRI)apparatus, are conducted based on imaging order information via aninformation management apparatus. Medical images obtained by imaging aredigitized and the digitized medical images are stored and managedtogether with additional information, such as patient information andexamination information, in an image server.

The medical images and additional information are managed on a networkthat connects the modalities and the image server, so that, for example,any correction made on information in either the modalities or the imageserver needs to be managed to maintain the consistency of theinformation therebetween. In relation to this, Japanese PatentApplication Laid-Open No. 2009-125137 discusses a method for updatingadditional information added to a medical image with new information andreflecting the updated information in an image server so as to managethe inconsistency in additional information that is caused wheninformation is updated in one of the image server and a client terminal.

SUMMARY

A medical image processing apparatus includes an obtaining unitconfigured to obtain a medical image based on imaging order information,and a determination unit configured to determine, using parametersobtained by machine learning, consistency between the imaging orderinformation and the medical image.

Further features of various embodiments will become apparent from thefollowing description of exemplary embodiments with reference to theattached drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram illustrating a schematic configuration of amedical image processing apparatus according to an exemplary embodiment.

FIG. 2 is a flowchart illustrating a processing procedure according to afirst exemplary embodiment.

FIG. 3 illustrates a display example on a display unit according to thefirst exemplary embodiment.

FIG. 4 illustrates a display example on the display unit according tothe first exemplary embodiment.

FIG. 5 is a flowchart illustrating a processing procedure according to asecond exemplary embodiment.

FIG. 6 illustrates a display example on a display unit according to thesecond exemplary embodiment.

FIG. 7 is a flowchart illustrating a processing procedure according to athird exemplary embodiment.

FIG. 8 illustrates a display example on a display unit according to thethird exemplary embodiment.

FIG. 9 is a flowchart illustrating a processing procedure according to afourth exemplary embodiment.

FIG. 10 is a flowchart illustrating a processing procedure according toa fifth exemplary embodiment.

DESCRIPTION OF THE EMBODIMENTS

For example, a medical image processing system for use in the field ofmedicine needs to be operated by a technician specializing in operatingthe medical image processing system. The technician who obtains medicalimages may not be the same person as a doctor who conducts anexamination using medical images.

For example, the technician operates the medical image processing systembased on a request (order) from the doctor who has examined a patient tocollect medical images, perform image processing on the medical images,and transfer the medical images to a server. In this case, for example,if there is inconsistency between the order and obtained medical images,the workflow can be impaired by re-obtaining medical images. In somecases, the inconsistency therebetween can be noticed only after thepatient has gone home.

One aspect of an exemplary embodiment is to accurately determineconsistency between imaging order information (imaging information thathas been ordered by a health-care provider, such as a doctor) and amedical image obtained based on the imaging order information.

To solve the above-described issues, a medical image processingapparatus according to an aspect of the present exemplary embodimentincludes an image obtaining unit configured to obtain a medical imagebased on imaging order information, and a determination unit configuredto determine consistency between the imaging order information and themedical image using a parameter obtained by machine learning.

The medical image processing apparatus according to the presentexemplary embodiment can accurately determine consistency betweenimaging order information and a medical image obtained based on theimaging order information.

Exemplary embodiments will be described below with reference to thedrawings. While the following exemplary embodiments are described usingan example where a medical image processing system for radiologicalimaging is used, some embodiments are applicable to medical imageprocessing systems using other modalities, such as a computed tomography(CT) apparatus, a magnetic resonance imaging (MRI) apparatus, and anultrasonic apparatus. Some embodiments are also applicable to medicalimage processing systems using a combination of various types ofmodalities.

Outline Configuration

Some embodiments are applied to, for example, a medical image processingapparatus 100 illustrated in FIG. 1 . The medical image processingapparatus 100 includes an image obtaining unit 101, a central processingunit (CPU) 102, a storage unit 103, a main memory 104, an operation unit105, a display unit 106, and a consistency determination unit 107. Theimage obtaining unit 101, the CPU 102, the storage unit 103, the mainmemory 104, the operation unit 105, the display unit 106, and theconsistency determination unit 107 are connected to each other via a CPUbus 130 and are configured to mutually transmit and receive datatherebetween.

In the medical image processing apparatus 100, the main memory 104functions as a working memory used for processing in the CPU 102. TheCPU 102 controls the overall operation of the medical image processingapparatus 100 using the main memory 104 in accordance with an operationinput via the operation unit 105 and parameters stored in the storageunit 103. With this configuration, the medical image processingapparatus 100 operates as follows.

First, when an imaging instruction is input by a user using theoperation unit 105 based on imaging order information transmitted froman information management system (not illustrated), the CPU 102transmits this imaging instruction to the image obtaining unit 101.Examples of an information management system used in a hospital includea hospital information system (HIS) that manages information includingpatient information, such as a patient name and a patient identification(ID), and examination request information including a date and time ofan examination and imaging content. Examples of the informationmanagement system include a radiology information system (RIS) thatmanages information, such as patient information and examination requestinformation, particularly, in a radiology department. The term “imagingorder information” refers to a unit of information about an examinationordered by a doctor, and includes patient information, a (scheduled)date and time of imaging, a designated examination method (X-rayexamination, CT examination, etc.), and an area, a direction, and anorientation of an imaging target based on doctor's findings.

An example where an X-ray examination is designated as an examinationmethod ordered by a doctor will be described below. In this case, themedical image processing apparatus 100 is an X-ray diagnostic apparatusand the image obtaining unit 101 is an X-ray image obtaining unit. Theimaging order information includes information used for X-ray imaging,such as the type (upright type, lying type, portable type, etc.) of anX-ray detection apparatus used for imaging, the orientation (imagingarea, direction, etc.) of a patient, and X-ray imaging conditions (tubevoltage, tube current, and presence or absence of a grid, etc.).

Upon receiving the imaging instruction, the image obtaining unit 101controls a radiation generation unit and a radiation detector to executeradiographic imaging. In radiographic imaging, the radiation detectorfirst detects a radiation beam that is irradiated from the radiationgeneration unit and is transmitted through a subject while beingattenuated, and the image obtaining unit 101 obtains a signalcorresponding to the intensity of the radiation beam as image data. Theimage data is sequentially transferred to the main memory 104 and theconsistency determination unit 107 via the CPU bus 130.

The consistency determination unit 107 inputs the transferred image datato an inference unit 108, outputs a result of inference processing, anddetermines consistency between the imaging order information and theimage data based on the inference result.

Specifically, for example, in the case of determining the consistency onthe imaging area included in the imaging order information, theinference unit 108 performs inference processing on the imaging areabased on the image data, and the consistency determination unit 107determines whether the inferred imaging area matches information on theimaging area included in the imaging order information.

While the imaging area is described above as an example of the imagingorder information, the imaging order information on which theconsistency determination unit 107 performs determination is not limitedto the imaging area. The consistency determination unit 107 maydetermine the consistency with respect to one or more pieces ofinformation. The imaging order information, the image data, and theinference processing result are transferred to, for example, the storageunit 103 and the display unit 106 via the CPU bus 130. The storage unit103 stores the transferred imaging order information and the image databased on the inference processing result. The display unit 106 displaysinformation based on the transferred imaging order information, imagedata, and inference processing result. Display of information on thedisplay unit 106 is performed by a display control unit (notillustrated) included in the medical image processing apparatus 100. Inother words, the display control unit corresponds to an example of adisplay control unit that displays a determination result output fromthe determination unit and an instruction for an operator based on thedetermination result on the display unit 106.

The user checks the information that is based on the displayed imagingorder information, image data, and inference processing result, andissues an operation instruction, as needed, via the operation unit 105.

The consistency determination unit 107 further includes a comparisonunit 109 that performs consistency determination processing using arule-based comparison algorithm.

The consistency determination unit 107 further includes a learning unit110 and a verification unit 111. The learning unit 110 updatesparameters for the inference unit 108. The verification unit 111verifies the accuracy of inference using the parameters. The medicalimage processing apparatus 100 further includes a data control unit 112that controls operations of the learning unit 110 and the verificationunit 111.

As a first exemplary embodiment, a description will be given of a casewhere X-ray imaging is performed using the X-ray diagnostic apparatusbased on imaging order information to determine consistency between anobtained X-ray image and the imaging order information, and if there isinconsistency between the X-ray image and the imaging order information,an alert is issued, with reference to FIG. 2 .

Processing Flow

In step S201, the image obtaining unit 101 obtains an X-ray image basedon information that is derived from the transmitted imaging orderinformation and that is necessary to obtain the X-ray image. Examples ofthe information necessary to obtain the X-ray image include the type(upright type, lying type, portable type, etc.) of the X-ray detectionapparatus to be used, the orientation (imaging area, direction, etc.) ofa patient, and X-ray imaging conditions (tube voltage, tube current,presence or absence of a grid, etc.). The operator of the X-raydiagnostic apparatus makes settings for the X-ray diagnostic apparatusand performs positioning of the patient based on these pieces ofinformation, and then presses an X-ray irradiation switch. The imageobtaining unit 101 then obtains an X-ray image.

In this case, the operator interprets the type of the X-ray detectionapparatus to be used and the X-ray imaging conditions based on thereceived imaging order information, and makes settings for the imageobtaining unit 101. In some embodiments, an image obtaining control unit113 controls the image obtaining unit 101 so that the consistencydetermination is performed before X-ray irradiation and the X-rayirradiation is prevented from being performed in a state ofinconsistency caused by the operator's interpretation error. In otherwords, the image obtaining control unit 113 corresponds to an example ofan image obtaining control unit configured to control the imageobtaining unit not to obtain an image if there is inconsistency betweenthe imaging order information and the type of the image obtaining unitand parameter settings. Alternatively, the image obtaining control unit113 controls the image obtaining unit 101 to interpret, receive, andautomatically set the type of the X-ray detection apparatus and X-rayimaging conditions based on the imaging order information before X-rayirradiation. In other words, the image obtaining control unit 113corresponds to an example of the image obtaining control unit configuredto perform control related to the type of the image obtaining unit andparameter settings based on the imaging order information.

In step S202, the consistency determination unit 107 determinesconsistency between the X-ray image obtained by the image obtaining unit101 and the imaging order information. In this case, the consistencydetermination unit 107 is configured to perform determination processingusing an inference model (inference unit 108) that is obtained bymachine learning and includes updatable parameters.

In other words, the consistency determination unit 107 corresponds to anexample of the determination unit configured to determine consistencybetween the imaging order information and the medical image using aparameter obtained by machine learning.

For example, if the orientation of a patient is set as a consistencydetermination processing target, data is supplied to the X-raydiagnostic apparatus as X-ray image data only after image obtainingusing X-ray irradiation, unlike the case where the type of the X-raydetection apparatus and X-ray imaging conditions are consistencydetermination processing targets. In the consistency determination unit107, when X-ray image data is input, the inference unit 108 ispreliminarily subjected to machine learning to infer the orientation(imaging area, direction, etc.) of the patient. Because X-ray images ofvarious patients with various orientations are input and outputs need tomeet various demands from the operator, the inference unit 108 can usean inference device that is obtained by deep learning and uses anautomatically designed feature amount instead of a manually designedfeature amount. In other words, the consistency determination unit 107can include the inference unit 108 that has been subjected to learningprocessing using a deep learning algorithm to output an inferenceprocessing result about consistency between the medical image receivedas input and the imaging order information.

In a configuration in which the image obtaining control unit 113 isomitted, consistency determination processing and control processing bythe image obtaining unit 101 before X-ray irradiation cannot beperformed, so that the type of the X-ray detection apparatus and X-rayimaging conditions are to be determined based on X-ray image data. Inthis case, the consistency determination unit 107 performs theconsistency determination processing. However, the method forconsistency determination processing is not limited to deep learning.The consistency determination unit 107 can also perform the consistencydetermination processing by, for example, including the type of theX-ray detection apparatus to be used and X-ray imaging conditions inX-ray image data as additional information and using the comparison unit109 that simply compares the additional information with informationinterpreted based on the imaging order information. In other words,additional information may be added to a medical image, and theconsistency determination unit 107 may determine consistency betweenimaging order information and the medical image by comparing theadditional information with the imaging order information. Instead ofadding additional information to X-ray image data, the consistencydetermination processing can also be performed using an image analysisunit 114 that performs analysis processing based on a rule-basedanalysis technique using a contrast of the X-ray image, an intensity ofa pixel value and a specific frequency spectrum that substitute theX-ray imaging conditions, such as a tube voltage, a tube current, andinformation indicating the presence or absence of a grid. In otherwords, the consistency determination unit 107 may determine consistencybetween imaging order information and a medical image by analyzing themedical image using a rule-based technique. An inference model can alsobe obtained by machine learning. Which one of the above-describedmethods is to be employed can be determined depending on the cost,performance, necessity of updating by machine learning, or the like. Forexample, the consistency determination unit 107 may be configured by acombination of the comparison unit 109 for the type of the X-raydetection apparatus, the inference unit 108 for the orientation of thepatient, and the image analysis unit 114 for X-ray imaging conditions.In other words, the consistency determination unit 107 can change thedetermination method based on the consistency determination processingtarget.

In step S203, the display unit 106 displays the X-ray image obtained bythe image obtaining unit 101, the determination result output from theconsistency determination unit 107, or an instruction for the operatorthat is based on the determination result. For example, if thedetermination result indicates consistency between the imaging orderinformation and the obtained X-ray image, only the X-ray image is to bedisplayed. However, if the determination result indicates that there isinconsistency between the imaging order information and the obtainedX-ray image, the X-ray image, the imaging order information, and thedetermination result are to be displayed, and an instruction for theoperator is also to be displayed to ask the operator to performre-imaging, as illustrated in FIG. 3 .

Also, a more detailed inference processing result may be displayed.Specifically, the display unit 106 can display the determination resultoutput from the determination unit for each of a plurality of differentitems.

For example, the inference processing result can indicate that the typeof the X-ray detection apparatus and the orientation (imaging area,direction, etc.) of the patient in the imaging order information arecorrect, while the X-ray imaging conditions are incorrect as illustratedin FIG. 4 . Displaying such a detailed inference processing resultallows the operator to determine, for example, that, if the patient'sorientation that needs to reliably satisfy the consistency in terms ofdiagnosis is correct, the errors in the X-ray imaging conditions can beallowed as long as the diagnostic performance can be maintained. It alsoallows the operator to determine that X-ray imaging conditions adjustedon the spot can be used more suitably than the X-ray imaging conditionsinstructed in the order depending on the body type or the like of thepatient.

Further, a consistency determination item may be added depending on atarget area. Specifically, the consistency determination unit 107 canperform the consistency determination processing on a plurality ofdifferent items depending on an imaging area of a medical image. Forexample, in a chest image, the items can include an item indicatingwhether the size of an X-ray detector to be used is sufficiently largeto depict the entire area of a lung field, and an item indicatingwhether positioning of the imaging area is accurately performed. Foranother example, in an order for obtaining an image of a finger bonefracture or the like, an item indicating whether the right and leftsides of an imaged area are correct and an item indicating whether aregion of interest is located at a desired position can be set asdetermination items.

The medical image processing apparatus 100 executes a series ofprocessing as described above.

According to the present exemplary embodiment described above, theconsistency between imaging order information and a medical imageobtained based on the imaging order information can be accuratelydetermined.

As a second exemplary embodiment, a description will be given of a casewhere imaging management processing is performed using consistencydetermination processing when a plurality of times of image capturingusing the X-ray diagnostic apparatus is instructed in imaging orderinformation, with reference to FIG. 5 . This corresponds to an imagingsupport function, for example, in the case of imaging with a pluralityof orientations in one imaging order, such as X-ray imaging of fourlimbs, or in the case of changing an imaging sequence flexibly dependingon the state of a patient. Even when imaging is performed while changingthe imaging sequence, the patient's orientations at which images havebeen obtained can be recognized by performing consistency determinationprocessing on the obtained images, and patient's orientations at whichimages have not been obtained yet are presented via the display unit 106to thereby support the operator.

Processing Flow

In step S501, the image obtaining unit 101 obtains an X-ray image basedon information that is derived from the transmitted imaging orderinformation and used to obtain the X-ray image. Examples of theinformation used to obtain the X-ray image include the type (uprighttype, lying type, portable type, etc.) of the X-ray detection apparatusto be used, the orientation (imaging area, direction, etc.) of apatient, and X-ray imaging conditions (tube voltage, tube current,presence or absence of a grid, etc.). The operator of the X-raydiagnostic apparatus makes settings for the X-ray detection apparatusand performs positioning of the patient based on these pieces ofinformation, and then presses the X-ray irradiation switch. The imageobtaining unit 101 then obtains an X-ray image.

In this case, the imaging order information instructs to obtain aplurality of images, such as images of a front surface and a sidesurface of the chest of the patient, and the operator of the X-raydiagnostic apparatus determines an imaging operation to be subsequentlyperformed from among the imaging operations that have not been performedyet flexibly depending on the state of the patient.

In step S502, the consistency determination unit 107 determinesconsistency between the X-ray image obtained by the image obtaining unit101 and the imaging order information. In this case, the determinationby the consistency determination unit 107 includes determination usingthe inference model (inference unit 108) that is obtained by machinelearning and can be updated, which is the most characteristicconfiguration in the present exemplary embodiment.

In the present exemplary embodiment, for example, when the imaging orderinformation instructs to obtain two images, i.e., images of the frontsurface and the side surface of the chest, it is determined which one ofthe front surface and the side surface the X-ray image currentlyobtained by the image obtaining unit 101 corresponds to. Specifically,the consistency determination unit 107 inputs X-ray image data to theinference unit 108, and the inference unit 108 outputs an inferenceprocessing result indicating which one of the front surface and the sidesurface of the chest corresponds to the input X-ray image data.Specifically, if the imaging order information includes an order forobtaining a plurality of images, the consistency determination unit 107can determine which one of the plurality of images corresponds to themedical image obtained by the image obtaining unit. The inference unit108 is obtained in advance by machine learning. Because the imaginginstruction in imaging order information varies depending on thehospital where the medical image processing apparatus 100 is used, theinference unit 108 can include parameters obtained by deep learninginstead of using the rule-based technique.

In step S503, the display unit 106 displays the X-ray image obtained bythe image obtaining unit 101, the determination result output from theconsistency determination unit 107, and the instruction for the operatorthat is based on the determination result. For example, if thedetermination result indicates that one of a plurality of imagingoperations requested in the imaging order information has been carriedout, an instruction to obtain a not-yet-obtained image is displayed asillustrated in FIG. 6 . In other words, if the imaging order informationincludes an order for obtaining a plurality of images, the display unit106 can display an instruction to obtain a not-yet-obtained image amongthe plurality of images requested in the order, based on an output fromthe determination unit.

The above-described processing is repeatedly performed until all theimages requested in the imaging order information are obtained, therebymaking it possible to prevent imaging with the same orientation andprevent omission of some of imaging operations. This leads to animprovement in workflow.

As a third exemplary embodiment, a description will be given of a casewhere X-ray imaging is performed based on imaging order informationusing the X-ray diagnostic apparatus and the obtained X-ray image andimaging order information are stored as data for learning or data forverification according to the consistency determination result, withreference to FIG. 7 .

Processing Flow

In step S701, the image obtaining unit 101 obtains an X-ray image basedon information that is derived from the transmitted imaging orderinformation and used to obtain the X-ray image. Examples of theinformation used to obtain the X-ray image include the type (uprighttype, lying type, portable type, etc.) of the X-ray detection apparatusto be used, the orientation (imaging area, direction, etc.) of apatient, and X-ray imaging conditions (tube voltage, tube current,presence or absence of a grid, etc.). The operator of the X-raydiagnostic apparatus makes settings for the X-ray diagnostic apparatusand performs positioning of the patient based on these pieces ofinformation, and then presses the X-ray irradiation switch. The imageobtaining unit 101 then obtains an X-ray image.

In step S702, the consistency determination unit 107 determinesconsistency between the X-ray image obtained by the image obtaining unit101 and the imaging order information. In this case, the determinationby the consistency determination unit 107 includes determination usingthe inference model (inference unit 108) that is obtained by machinelearning and can be updated, which is the most characteristicconfiguration in the present exemplary embodiment.

In step S703, the display unit 106 displays the X-ray image obtained bythe image obtaining unit 101, the determination result output from theconsistency determination unit 107, or the instruction for the operatorthat is based on the determination result. For example, if thedetermination result indicates consistency between the imaging orderinformation and the obtained X-ray image, only the X-ray image is to bedisplayed. However, if the determination result indicates that there isinconsistency between the imaging order information and the obtainedX-ray image, the X-ray image, the imaging order information and thedetermination result are to be displayed, and a re-imaging instructionfor the operator is to be also displayed.

In step S704, the display unit 106 displays an option to accept a useroperation regarding storage, specifically regarding whether to store theimaging order information together with the X-ray image as data forlearning or data for verification in the storage unit 103, asillustrated in FIG. 8 .

Assume that the option displayed here varies depending on thedetermination result output from the consistency determination unit 107.The displayed options includes an option “re-image” for the case wherethe determination result indicates that the imaging order informationdoes not match the captured image and an option “match” for the casewhere the determination result is different and the imaging ordermatches the captured image. The case where “the determination result isdifferent” corresponds to a case where the inference unit 108 includedin the consistency determination unit 107 has failed to correctlyperform inference processing on the currently input image. In otherwords, this image and the imaging order information including a correctanswer can be provided as data for learning to the inference unit 108.Accordingly, when the option “match” is selected, the display cantransition to a screen for confirming the operator's intention to storethis image as data for learning. Even in a situation where the option“re-image” is selected, that is, even when the determination result iscorrect, it can be assumed that the operator desires to store theimaging order information and the image as data for verification. Inthis case, when the option “re-image” is selected, the display maytransition to a screen for confirming the operator's intention to storethis image as data for verification. In other words, the display unit106 can display an option to receive an input from the operatorregarding storage of the imaging order information and the medical imagebased on the determination result output from the determination unit.Further, the display unit 106 can display information indicating thatthe imaging order information and the medical image are sorted as datafor learning in a case where the determination result output from thedetermination unit is incorrect. If the determination result is correct,the display unit 106 can display information indicating that the imagingorder information and the medical image are sorted as data forverification. Further, the storage unit 103 can sort and store theimaging order information and the medical image as data for learning ordata for verification according to an input from the operator via theoperation unit.

The above-described screen transition that occurs every time imaging isperformed is troublesome for the operator and leads to a reduction inworking efficiency. Therefore, it may be desirable to automatically sortand store the imaging order information and the medical image as data“for verification” or data “for learning” after selecting the option“re-image” or “match”, without displaying the screen for confirming theoperator's intention. Alternatively, the screen transition may beperformed at a preset frequency, for example, once every ten times. Inother words, the display unit 106 can display a screen for receiving anoperation to store the imaging order information and the medical imageat a predetermined frequency.

Also, in the case of automatically sorting and storing the imaging orderinformation and the medical image as data “for verification” or data“for learning”, for example, the imaging order information and themedical image can be sorted into data “for verification” and data “forlearning” at a ratio of 9:1 when the option “re-image” is selected, andthe imaging order information and the medical image can be sorted intodata “for verification” and data “for learning” at a ratio of 1:9, forexample, when the option “match” is selected. It is desirable to makethe sorting ratio adjustable by the hospital or the operator. Thestorage unit 103 can store the sorted imaging order information andmedical image while sorting the imaging order information and themedical image into data for learning and data for verification at apredetermined ratio.

In step S705, the storage unit 103 sorts and stores the imaging orderinformation and the captured image as data “for verification” or data“for learning” based on the option selected via the operation unit 105.As described above, if the order information is different from theconsistency determination result, the image is desirably registered asan image for learning. However, since data “for verification” is needed,it is desirable to make the sorting ratio settable at an adjustableratio instead of registering all the images as data “for learning”.

If all data on, for example, chest images that are captured at a highfrequency is stored, the data occupies a large part of the storagecapacity of the storage unit 103. Therefore, it is desirable to make thestorage ratio adjustable for each area. Specifically, data on an areawhere the imaging frequency is high is to be stored once every tentimes, and all data is to be store stored with respect to an area wherethe imaging frequency is low. In other words, the data control unit 112can set, for each imaging area of a medical image, the frequency ofdisplaying the screen for receiving an operation regarding storage inthe storage unit 103, or the sorting ratio at which the imaging orderinformation and the medical image are sorted as data for learning ordata for verification.

Further, since imaging based on the imaging order information varieswith time according to the level of proficiency, new data can be used asdata for learning and data for verification. Accordingly, the datastored in the storage unit 103 is discarded in sequence from the oldestdata so as to prevent the data from occupying the storage capacity.

Furthermore, it is desirable to provide a mechanism for deletinginappropriate data for use in learning, for example, data obtainedduring test-imaging or data obtained when imaging has failed, becausesuch inappropriate data may deteriorate the performance of the inferenceunit 108 in the learning process.

As a fourth exemplary embodiment, a description will be given oflearning processing to obtain a new parameter for the inference unit 108included in the consistency determination unit 107 using an X-ray imageand imaging order information stored as data for learning in the storageunit 103 with reference to FIG. 9 .

Processing Flow

In step S901, the data control unit 112 obtains an X-ray image andimaging order information stored as data for learning in the storageunit 103. If the orientation (imaging area, direction, etc.) of thepatient is set as an inference processing target of the inference unit108, the imaging order information used here is label informationindicating the orientation of a patient.

In step S902, the inference unit 108 performs inference processing usingthe X-ray image obtained by the data control unit 112 as input andoutputs an inference processing result.

In step S903, the learning unit 110 calculates a loss and updatesparameters for the inference unit 108 based on back propagation usingthe inference processing result output from the consistencydetermination unit 107 and the label information obtained from thestorage unit 103. This promotes optimization of the inference process bythe inference unit 108 such that the inference unit 108 infers the labelappended to the input image.

In step S904, the data control unit 112 repeatedly performs theabove-described learning step a predetermined number of times for apredetermined number of pieces of data at a predetermined timing, andstores the results in the storage unit 103 as new parameter candidatesconstituting the inference unit 108. In other words, the data controlunit 112 corresponds to an example of a data control unit that storesparameters that are updated using the imaging order information and themedical image obtained as data for learning from the storage unit, asnew parameter candidates for the determination unit.

Examples of the predetermined timing include a timing when new data forlearning is stored in the storage unit 103, a timing when apredetermined number of pieces of data for learning are accumulated, anda timing outside the reception hours in a hospital when the X-raydiagnostic apparatus is not used in clinical practice. The predeterminednumber of pieces of data and the predetermined number of times aregenerally determined depending on the type of a learning algorithm to beused and calculation resources to be operated, and are not particularlylimited.

According to the present exemplary embodiment, the inference unit 108can be updated by machine learning using the obtained image and theimaging order information. It is intended to improve the accuracy of theinference unit 108 by employing the configuration of the inference unit108 that can be customized to perform desired inference processing forindividual operated facilities, instead of implementing the inferenceunit 108 that is configured in advance to perform inference processingon any type of inference target. The inference unit 108 can be updatedby additional learning so that recognition, classification, regression,and the like in the inference unit 108 can be optimized for rulesdetermined based on an examination target and workflow for individualfacilities, thereby making it possible to construct a system that can beimproved as much as the system is used depending on the characteristicsof individual facilities.

Verification and Update

As a fifth exemplary embodiment, a description will be given of a casewhere the performance of a new inference parameter candidate is verifiedand the inference unit 108 is updated with reference to FIG. 10 .

Processing Flow

In step S1001, the data control unit 112 obtains an X-ray image andimaging order information stored as data for verification stored in thestorage unit 103. In a case where the orientation (imaging area,direction, etc.) of the patient is set as an inference processing targetof the inference unit 108, the imaging order information used here islabel information indicating the orientation of a patient specifically.

In step S1002, the inference unit 108 performs inference processingusing the X-ray image obtained by the data control unit 112 as input andoutputs an inference processing result.

In step S1003, the verification unit 111 evaluates the inferenceaccuracy using the inference processing result output from theconsistency determination unit 107 and the label information obtainedfrom the storage unit 103. The above-described steps are repeatedlyperformed on the data stored as data for verification in the storageunit 103, and the inference accuracy is calculated as the performance ofan inference parameter candidate.

In step S1004, the data control unit 112 calculates the above-describedinference accuracy at a timing when a new parameter candidateconstituting the inference unit 108 is stored in the storage unit 103,and updates the inference unit 108 with the new parameter candidate ifthe inference accuracy is more than a predetermined value. In otherwords, the data control unit 112 can select a parameter from among newparameter candidates based on the evaluation result of the inferenceaccuracy calculated by the verification unit, and can update thedetermination unit using the selected parameter.

The predetermined value is, for example, the inference accuracy obtainedwhen steps S1001 to S1003 are performed on the inference unit 108 beforeupdating. In this case, it is possible to implement the inference unit108 that is automatically updated when the performance of a newparameter candidate that is higher than the performance of the currentlyused parameter can be achieved.

Alternatively, the above-described operation of the data control unit112 may be performed when the operator issues an instruction to carryout the operation via the operation unit 105. In this case, theinference accuracy of the new parameter candidate is presented to theoperator via the display unit 106, and the operator determines whetherto perform updating based on the inference accuracy. If the operatordetermines to perform updating, the operator updates the inference unit108 via the operation unit 105. This makes it possible to safely updatethe inference unit 108 based on the determination by the operator.

Alternatively, the apparatus can be configured to perform control basedon determination by a manufacturer of the apparatus instead ofdetermination by the operator.

In this case, a service engineer collects data, performs learning at themanufacturer, verification, and introduction of new parameter candidatesin a hospital. Alternatively, a cloud technology can be desirably usedso as to execute data collection via a network, verification on thecloud, and updating based on an instruction remotely issued by themanufacturer.

It may also be desirable to store and manage the parameter candidatesand update history, and if a problem occurs, the data is to beimmediately restored to the original state before updating. This makesit possible to safely update the inference unit 108.

Various Modifications and Changes

While exemplary embodiments have been described above, some embodimentsare not limited to the above-described exemplary embodiments. Variousmodifications and changes can be made within the scope of thedisclosure.

The medical image processing apparatus according to any one of theexemplary embodiments described above may be implemented as a singleapparatus, or may be configured to execute the above-describedprocessing using a combination of a plurality of apparatuses that arecommunicably connected to each other. These configurations are alsoincluded in the exemplary embodiments of the present disclosure. Theabove-described processing may be executed by a common server apparatusor a server group. The plurality of apparatuses constituting the medicalimage processing apparatus can be configured to communicate with eachother at a predetermined communication rate, and are not required to belocated within the same facilities or the same country.

Other Embodiments

Some embodiment(s) can also be realized by a computer of a system orapparatus that reads out and executes computer-executable instructions(e.g., one or more programs) recorded on a storage medium (which mayalso be referred to more fully as a ‘non-transitory computer-readablestorage medium’) to perform the functions of one or more of theabove-described embodiment(s) and/or that includes one or more circuits(e.g., application specific integrated circuit (ASIC)) for performingthe functions of one or more of the above-described embodiment(s), andby a method performed by the computer of the system or apparatus by, forexample, reading out and executing the computer-executable instructionsfrom the storage medium to perform the functions of one or more of theabove-described embodiment(s) and/or controlling the one or morecircuits to perform the functions of one or more of the above-describedembodiment(s). The computer may comprise one or more processors (e.g.,central processing unit (CPU), micro processing unit (MPU)) and mayinclude a network of separate computers or separate processors to readout and execute the computer-executable instructions. Thecomputer-executable instructions may be provided to the computer, forexample, from a network or the storage medium. The storage medium mayinclude, for example, one or more of a hard disk, a random-access memory(RAM), a read only memory (ROM), a storage of distributed computingsystems, an optical disk (such as a compact disc (CD), digital versatiledisc (DVD), or Blu-ray Disc (BD)™), a flash memory device, a memorycard, and the like.

While the present disclosure has described exemplary embodiments, it isto be understood that some embodiments are not limited to the disclosedexemplary embodiments. The scope of the following claims is to beaccorded the broadest interpretation so as to encompass all suchmodifications and equivalent structures and functions.

This application claims priority to Japanese Patent Application No.2021-162902, which was filed on Oct. 1, 2021 and which is herebyincorporated by reference herein in its entirety.

What is claimed is:
 1. A medical image processing apparatus comprising:an obtaining unit configured to obtain a medical image based on imagingorder information; and a determination unit configured to determine,using parameters obtained by machine learning, consistency between theimaging order information and the medical image.
 2. The medical imageprocessing apparatus according to claim 1, further comprising a controlunit configured to control the obtaining unit not to obtain the medicalimage in a case where there is inconsistency between the imaging orderinformation and settings for the parameters related to a type of themedical image and obtaining of the medical image.
 3. The medical imageprocessing apparatus according to claim 2, wherein the control unitperforms control related to the type of the medical image and thesettings for the parameters based on the imaging order information. 4.The medical image processing apparatus according to claim 1, wherein,using the medical image as an input, the determination unit selects,based on a consistency determination target, any one of a plurality ofdetermination methods including at least a determination method thatuses the parameters obtained using a deep learning algorithm, theplurality of determination methods including at least one of adetermination method that compares the imaging order information andadditional information added to the medical image and a determinationmethod that analyzes the medical image using a rule-based technique. 5.The medical image processing apparatus according to claim 1, furthercomprising a display control unit configured to cause a display unit todisplay a determination result of the consistency, wherein thedetermination unit determines the consistency on a plurality ofdifferent items based on an imaging area of the medical image, andwherein the display control unit causes the display unit to display adetermination result of the consistency for each of the plurality ofdifferent items.
 6. The medical image processing apparatus according toclaim 1, further comprising a display control unit configured to cause adisplay unit to display an instruction for an operator based on adetermination result of the consistency, wherein in a case where theimaging order information includes an order for obtaining a plurality ofmedical images, the determination unit determines which one of theplurality of medical images corresponds to the medical image obtained bythe obtaining unit, and wherein the display control unit causes thedisplay unit to display an instruction to obtain a not-yet-obtainedimage among the plurality of medical images.
 7. The medical imageprocessing apparatus according to claim 1, further comprising: a storageunit configured to store the imaging order information and the medicalimage; and an operation unit configured to receive an input from anoperator, wherein the imaging order information and the medical imageare sorted into data for learning and data for verification according tothe input from the operator via the operation unit, and the sortedimaging order information and medical image are stored in the storageunit.
 8. The medical image processing apparatus according to claim 1,further comprising a display control unit configured to cause a displayunit to display an option to receive an input from an operator regardingstorage of the imaging order information and the medical image, based ona determination result from the determination unit.
 9. The medical imageprocessing apparatus according to claim 8, wherein the display controlunit causes the display unit to display information indicating that theimaging order information and the medical image are sorted as data forlearning in a case where the determination result from the determinationunit is incorrect and the imaging order information and the medicalimage are sorted as data for verification in a case where thedetermination result from the determination unit is correct.
 10. Themedical image processing apparatus according to claim 7, furthercomprising a control unit configured to control display for receiving anoperation regarding the storage at predetermined frequency.
 11. Themedical image processing apparatus according to claim 10, wherein thecontrol unit is configured to control the imaging order information andthe medical image to be stored in the storage unit by sorting theimaging order information and the medical image into data for learningand data for verification at a predetermined ratio, is configured toset, for each imaging area of the medical image, a frequency at whichthe display for receiving the operation regarding storage in the storageunit is performed and a ratio at which the imaging order information andthe medical image are sorted into data for learning and data forverification, or is configured to store, into the storage unit, theparameters updated with the imaging order information and the medicalimage for learning obtained from the storage unit as new parametercandidates.
 12. The medical image processing apparatus according toclaim 11, further comprising a verification unit configured to evaluatean accuracy of determination made by the determination unit includingthe new parameter candidates, wherein the control unit stores, into thestorage unit, the new parameter candidates for the determination unitand a determination accuracy evaluation result of the verification unitusing the imaging order information and the medical image forverification obtained from the storage unit.
 13. The medical imageprocessing apparatus according to claim 12, wherein the control unitselects a parameter from among the new parameter candidates based on thedetermination accuracy evaluation result of the verification unit, andupdates the determination unit with the selected parameter.
 14. Themedical image processing apparatus according to claim 12, wherein thecontrol unit selects a parameter from among the new parameter candidatesbased on an instruction issued by an operator via the operation unit,and updates the determination unit with the selected parameter.
 15. Themedical image processing apparatus according to claim 1, furthercomprising: a display control unit configured to cause a display unit todisplay a determination result from the determination unit; and acontrol unit configured to receive an input from an operator regardingstorage of the imaging order information and the medical image to updatethe parameters by machine learning.
 16. The medical image processingapparatus according to claim 1, wherein the imaging order information isa unit of information about an examination ordered by a doctor, andincludes at least one of patient information, a date and time ofimaging, a designated examination method, and an area, a direction, andan orientation of an imaging target based on doctor's findings.
 17. Amedical image processing method, comprising: obtaining a medical imagebased on imaging order information; and determining, using parametersobtained by machine learning, consistency between the imaging orderinformation and the medical image.
 18. A non-transitorycomputer-readable storage medium storing instructions for causing acomputer to execute a method comprising: obtaining a medical image basedon imaging order information; and determining, using parameters obtainedby machine learning, consistency between the imaging order informationand the medical image.